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Educational Complexity and Professional Development: 
Teachers’ Need for Metacognitive Awareness 
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Abstract 

The study was designed to investigate technology and engineering teachers’ 
metacognitive awareness during specific established teacher practices. The study 
had a sample size of 18. There were six participants in three groups. Group 1 
consisted of teachers that actively participated in Transforming Teaching 
through Implementing Inquiry (T2I2) professional development program. Group 
2 consisted of teachers that were selected for but did not actively participate in 
T2I2 professional development system. Group 3 consisted of teachers that 
completed the National Board for Professional Teaching Standards professional 
development program. To measure the metacognitive awareness of each group, 
a semi-structured open-ended interview was used. The interviews were analyzed 
by two independent coders using a coding rubric. The coded interviews 
established the phenomenological metacognitive awareness of each group. 

Keywords: Professional development, metacognitive awareness, technology and 
engineering education 

Metacognitive awareness is the ability to recognize and regulate one’s own 
thinking in real time. Metacognitive awareness is the term used to describe an 
individual’s ability to detail their knowledge and regulation of cognition 
(Schraw & Dennison, 1994). Knowledge of cognition and regulation of 
cognition are two predominant components of metacognition. Examples of 
metacognitive subcomponents under knowledge of cognition include 
declarative, procedural, and conditional knowledge, and examples of 
metacognitive subcomponents under regulation of cognition include planning, 
monitoring, organizing (information management), debugging, and evaluating 
(Schraw, 2001; Schraw & Dennison, 1994). 

The term declarative knowledge refers to the knowledge that a person has 
about their cognitive strategies, skills, and abilities (Schraw, 2001). A person with 
declarative knowledge knows what impacts their learning and the learning of others 
and what they do and do not know. A person with declarative knowledge knows 
strategies that can be used to increase performance for completing tasks. The term 
procedural knowledge refers to a person’s knowledge about how to use strategies 
and techniques to increase performance and accomplish cognitive tasks (Schraw, 
2001). A person with procedural knowledge will complete tasks by sequencing 
known strategies. The term conditional knowledge refers to the knowledge that a 
person has regarding when and why to use strategies for accomplishing tasks 
(Schraw, 2001). A person with conditional knowledge knows when and why to 
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present an idea and to use strategies for completing tasks. A person with conditional 
knowledge can rationalize the use of specific strategies and appropriately use 
strategies based on the scenario. 

The term planning is used to describe a person’s ability to select appropriate 
strategies, set goals, and allocate resources (Schraw, 2001). Planning relates to a 
person’s utilization of planning strategies, goal setting, and resources related to 
accomplishing tasks. The organizing subcomponent relates to the information 
management sub-process (Pucheu, 2008). Organizing is the use of cognitive 
strategies and techniques to manage information (Pucheu, 2008). Information 
management is the active process of organizing, elaborating, summarizing, and 
selectively focusing on important information for mental restructuring due to 
cognitive dissonance (Pucheu, 2008). During monitoring , a person assesses their 
cognition and strategy effectiveness (Schraw, 2001). When teachers are monitoring, 
they add the assessment of students’ thinking through verbal and nonverbal 
feedback to determine their own effectiveness. During the process of debugging a 
person uses strategies to identify and correct errors and assumptions about tasks and 
implemented strategies (Pucheu, 2008). The subcomponent of evaluating is the post 
hoc analysis of performance and strategy effectiveness (Pucheu, 2008). 

The varying complexity and duration of problems that teachers experience 
indicates their need for metacognition in terms of improved regulation of 
cognition (Hartman, 2001; Kramarski & Michalsky, 2009) and knowledge of 
cognition (Schraw, 2001; Wilson & Bai, 2010). Teachers that lack an awareness 
of their own cognitive abilities will have difficulty adapting in the constantly 
evolving educational environment (Kramarski & Michalsky, 2009; Lin, 

Schwartz, & Hatano, 2005). Prior research has established a link between 
teacher’s metacognitive skills and the effectiveness of their teacher practices 
(Georghiades, 2004; Gourgey, 1998; Hartman, 2001). Metacognitive awareness 
is also foundational in a person’s ability to learn (Ertmer & Newby, 1996). 
Teachers with higher levels of metacognitive awareness have improved learning 
capability and the ability to translate learning from professional development 
into classroom practices (Bransford, Brown, & Cocking, 2000; Ertmer & 

Newby, 1996; Pucheu, 2008). 

According to Bybee and Loucks-Horsley (2000), “professional 
development will provide the opportunities for technology teachers and other 
educators to learn what they need to know and be able to do as they assist 
students” (p. 32) with learning, but only if teachers have the “cognitive self- 
awareness necessary for the kinds of metacognitive capabilities required to 
transfer professional development training into effective classroom practices 
(Bransford, Brown & Cocking, 1999; Graber, 1998; Palincsar & Brown, 1984)” 
(Pucheu, 2008, p. 7). The term professional development (PD) refers to teachers’ 
improvement or growth of skills and knowledge, primarily with the aim of 
improving student achievement (Guskey, 2003; Loucks-Horsley, Hewson, Love, 
& Stiles, 1998). The literature presents numerous characteristics deemed 
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essential for increasing the effectiveness of PD (Guskey, 2003; Mundry, 2007; 
Smylie, Allensworth, Greenberg, Harris, & Luppescu, 2001; WestEd, 2000). 
Gusky (2003) pointed out that not all the literature agrees on specific 
characteristics of effective professional development. Mundry (2007) indicated 
that even without “an empirically-based consensus of what constitutes effective 
professional development (Guskey, 2003; Whitehurst, 2002) there is a 
knowledge base about learning to guide the design and implementation of 
teacher learning programs (Elmore, 2002; Loucks-Horsley, Love, Stiles, 

Mundry & Hewson, 2003)” (p. 1). The ability of professional development 
designers to determine teachers’ current level of metacognitive awareness and 
incorporate the further development of metacognitive awareness early on and 
throughout PD may impact teachers’ learning ability (Ertmer & Newby, 1996; 
Prytula, 2012), development of pedagogical knowledge (Kramarski & 

Michalsky, 2009), and ability to transfer learned content back into their 
classrooms (Pucheu, 2008). 

The research presented in this article aims to describe three different groups 
of teachers’ metacognitive awareness while performing established teacher 
practices. To describe each group’s metacognitive awareness, a semi-structured 
open-ended interview was used. The interview was designed to gather a 
perspective of participant’s metacognitive awareness during specific teacher 
practices. Participants were divide into groups based on their participation in one 
of two different PD programs. Even though the two PD programs offered 
metacognitive experiences, the development of metacognitive awareness was not 
a focal point of either program. The findings presented in this article suggest that 
each group’s prior level of metacognitive awareness was a factor in their 
successful completion of the PD program. Additionally, the findings suggest that 
each group’s metacognitive awareness related to established teacher practices 
was a factor in their ability to manage educational complexity by adapting these 
practices. 


Background 

This study was conducted in the context of a PD system called 
Transforming Teaching through Implementing Inquiry (T2I2), a project funded 
for 4 years by the National Science Foundation. 1 The project started in fall 2011 
with the development of a highly interactive cyberinfrastructure system for 
delivering research-based PD. The PD was designed for secondary technology 
and engineering teachers in Grades 6-12. There were five primary goals that the 
designers of T2I2 attempted to accomplish. The first goal for the T2I2 program 
was increasing the participating teachers’ ability to manage, monitor, adjust, and 
contribute in the learning environment. The second goal was to increase 


1 For more information about the creation of the T2I2 program, see Ernst, Clark, 
DeLuca, and Bottomley (2013). 
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teachers’ understanding of engineering design concepts and the ability to 
effectively teach these concepts. The third goal was to increase the teachers 
understanding of and ability to address student learning needs. The fourth goal 
was increasing teachers’ instructional abilities with the use of self-assessment. 
The final goal was to promote technology and engineering teachers’ attainment 
of National Board Certification by aligning T2I2 with National Boards for 
Professional Teaching Standards (NBPTS) in three key ways: (a) the primary 
goals of T2I2 align with the overall goals of NBPTS, (b) the 17 learning objects 
of T2I2 were aligned with the 13 Career and Technical Education standards 
within NBPTS, and (c) T2I2 aligns with NBPTS by using shorter versions of the 
same teacher artifacts used by NBPTS. 

Although the process of completing both the T2I2 and the NBPTS PD 
programs involves numerous metacognitive experiences, the importance of these 
experiences in developing metacognitive awareness is not communicated to 
participants or identified as the primary focus of either program. Because 
metacognitive awareness is involved throughout these experiences, it stands to 
reason that a teacher with already high levels of metacognitive awareness would 
have an easier time completing either PD program. 

Rationale 

The purpose of this research was to understand technology and engineering 
teachers’ level of metacognitive awareness in comparison to their participation 
and completion of either the T2I2 or the NBPTS PD program. This study was 
informed by research design, metacognitive, and PD literature. The literature 
indicated that metacognitive research often focuses on students’ thinking and 
regulation because of the belief that metacognitive awareness helps students 
become better, more self-regulated learners (Schraw, 2001). However, the focus 
of metacognitive research has been shifting from the students to teachers due to 
the belief that teachers who lack metacognitive awareness are unable to help 
students develop their metacognitive awareness (Kramarski & Michalsky, 2009; 
Prytula, 2012; Pucheu, 2008). The belief that metacognitively aware teachers 
can help students develop their metacognitive awareness has prompted interest 
in PD with varying levels of focus on metacognitive awareness (Prytula, 2012; 
Pucheu, 2008). 

A phenomenological approach was selected for this study to describe 
participant’s metacognitive awareness related to their teacher practices 
(Creswell, 2007). As a phenomenological study, the research design used 
qualitative, semi-structured, open-ended interviews to help understand each 
group’s metacognitive awareness during established teacher practices (Creswell, 
2007; Denzin & Lincoln, 1994). The metacognitive awareness interview was 
designed to gather a more complete perspective of the participant’s 
metacognitive awareness (Bryman, 2006; Creswell, 2007; Denzin & Lincoln, 
1994). The semi-structured and open-ended characteristics of the interview 


- 28 - 



Journal of Technology Education Vol. 29 No. 1, Fall 2017 


enabled participants to provide an uninfluenced depth to their responses and 
promoted emergence of themes and patterns stated by each group (Akturk & 
Sahin, 2011; Creswell, 2007; Denzin & Lincoln, 1994). Interviewing with broad 
open-ended questions to investigate metacognition was supported by the 
literature (Akturk & Sahin, 2011; Prytula, 2012). 

Questions from the metacognitive awareness interview asked the 
participants to detail their thinking during cognitive tasks including planning, 
monitoring, organizing, information management, debugging, and evaluating. 
Participants’ ability to describe their mental phenomenon was used to indicate a 
level of metacognitive awareness (Georghiades, 2004, p. 374). Literature 
supported listening to the interviewees with as few interruptions to their 
responses as possible (Creswell, 2007; Denzin & Lincoln, 1994). The findings 
were used to provide a detailed description of the three groups’ metacognition 
during common teacher practices. 


Method 

Instrumentation 

The metacognitive awareness interview (see Table 1) was modeled based 
on the components and subcomponents of the Metacognitive Awareness 
Inventory (MAI; Schraw & Dennison, 1994). The metacognitive awareness 
interview consisted of 11 questions, including three parts to Question 4 and two 
parts to Question 8. The metacognitive awareness interview questions were 
based on the regulation of cognition subcomponents from the MAI. Additional 
questions were infrequently used to guide the interviewee if the interviewer felt 
that they were straying from the focus of the question. The interviews were 
audio recorded and then transcribed for analysis. 

The transcribed interviews were analyzed by two trained coders using a 
coding rubric. The coding rubric was generated based on the metacognitive 
awareness subcomponent definitions from the MAI literature. The 
metacognitive awareness subcomponent definitions are the descriptions of 
evaluated items listed in the rubric. The coding rubric was organized according 
to the questions in the metacognitive awareness interview, the components of 
metacognitive awareness, and the levels of awareness. 
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Table 1 

Metacognitive Awareness Interview 


Question 

number 


Interview question 


1 Describe a method you use for planning instruction. 

2 Describe a method you use for planning assessment. 

3 Describe how you organize your instruction. 

4.1 Remembering that there are multiple types of information; 
describe how you inwardly interpret information? 

4.2 How do you inwardly interpret unspoken information from 
your students? 

4.3 How do you inwardly interpret a new technique learned from 
another teacher or a professional development? 

5 Describe how you monitor during instruction. 

6 Describe how you monitor during assessment. 

7 Describe how you adjust your teaching during a lesson. 

8.1 Do you self-evaluate after the instructional process? 

8.2 Describe how you self-evaluate after the instructional process. 


Participants 

The participants in this study were divided into three groups: (Group 1) 
teachers who actively participated and completed the T2I2 system, (Group 2) 
teachers who had been selected for but did not participate, completing less than 
11% of the T2I2 system, and (Group 3) teachers who had received National 
Board Certification in Career and Technical Education (CTE) from the NBPTS 
program. This study included technology and engineering teachers from three 
states: Illinois, North Carolina, and Virginia. Participants from these three states 
applied and were randomly selected for participation in the T2I2 pilot Years 1 
(2012-2013) or 2 (2013-2014). Teachers for Group 3 were also identified from 
these three states. The participants in Group 3 were identified through the 
NBPTS’s website. All National Board Certified Teachers, their states, counties, 
certification areas, and certification expiration dates were listed. Teachers on 
the list were filtered by state and by CTE certification. Participants in Group 3 
were identified as possible participants only if they had a currently valid 
NBPTS certification in CTE and were currently teaching technology and 
engineering education in one of the three states. A total of 73 state-certified 
technology and engineering teachers were initially identified for possible 
participation in this study; each teacher was connected to either the T2I2 or the 
NBPTS PD program. In order to have equal group sizes, 10 teachers from each 
group where randomly selected to participate. 
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The 30 teachers received an email explaining the study and requesting their 
participation. A total of 18 teachers, six from each group, responded with 
interest in participating with the study (Table 2). After participants made an 
informed decision to participate, each was assigned a unique identifying 
number. The participant’s interview recording and interview transcript were 
encrypted with the unique number. The participants were sent an email in 
which they were asked to specify a phone number, date, and, time for the 
interview. Interviews were conducted by phone at the specified date and time. 

Table 2 

Participant Group Demographics _ 

Variables 


Gender Grade level taught 

n (%) Experience n (%) 


Group 

Male 

Female 

n 

Mean 

(years) 

SD 

Middle 

school 

High 

school 

1 

5 (83.3%) 

1 (16.6%) 

6 

20 

11 

3 (50%) 

3 (50%) 

2 

4 (66.7%) 

2 (33.3%) 

6 

17.3 

8.5 

1 (16.7%) 

5 (83.3%) 

3 

3 (50%) 

3 (50%) 

6 

21.5 

8.2 

3 (50%) 

3 (50%) 

Combined 

12 (66.6%) 6 (33.3%) 

18 

19.6 

8.9 

7(38.9%) 11 (61.1%) 


Procedure 

The study was conducted over a 16-week period during fall 2014. The 
metacognitive awareness interview was used to gather a thorough perspective of 
each participant’s metacognitive awareness during common teaching practices. 
In an attempt to build rapport with the participant, the interviewer began the 
interview by asking questions about the participant’s background and experience 
related to teaching. Additional guiding questions were used at times to help the 
participant provide sufficient detail regarding aspects of their metacognitive 
awareness. The recorded interviews were transcribed and later coded by two 
independent and trained coders using a coding rubric (Table 3). 

The two coders were selected based on their experience in teaching and 
qualitative research. Coder 1 had taught for six years at nearly all levels K-16. 
Coder 1 had also been involved numerous times in the collection and analysis of 
qualitative research data. Coder 2 had spent 33 years teaching at the elementary 
and middle school level. Coder 2 had frequently performed qualitative data 
collection and analysis during their career. Prior to coding, personal and 
identifying information were removed from the transcripts. To aid the coding 
process, each transcript was bracketed into sections based on the interview 
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questions. The coders were not informed about any characteristics of the 
participant, including their assigned group. Each transcript was then axial coded 
using the coding rubric (Creswell, 2007). 

The coders had to be trained regarding axial coding. Axial coding is the 
process of coding using contextual themes to encompass a phenomenon like 
metacognitive awareness. The training consisted of the coders reading, 
discussing, and rationalizing their codes based on the rubric for two transcripts. 
The coders completed all coding with the trainer present. During the training, 
the coders would frequently ask the trainer about the appropriate code to assign 
for a transcript section. The trainer would encourage the use of the coding 
rubric. The trainer would ask the coders to compare the transcript section to the 
corresponding descriptors in the coding rubric and to indicate the appropriate 
code from the rubric based on the content of that section. The coders 
independently used the rubric to identify each participant’s level of knowledge 
and regulation of cognition based on their answers to the interview questions. 

Table 3 


Metacognitive Awareness Interview Coding Rubric 


Question 
focus: Sub¬ 
component 

Component 
of meta¬ 
cognitive 
awareness 

High level 
of meta¬ 
cognitive 
awareness 

(5) 

High to 
medium 
level of 
meta¬ 
cognitive 
awareness 

(4) 

Medium 
level of 
meta¬ 
cognitive 
awareness 

(3) 

Medium to 
low level of 
meta¬ 
cognitive 
awareness 
(2) 

Low level 
of 

metacogniti 

ve 

Awareness 

a) 

Declarative, 

Knowledge 

The 

The 

The 

The 

The 

procedural 


participant 

participant 

participant 

participant 

participant 

and 


describes a 

describes 3 

describes 2 

describes 1 

does not 

conditional 


strategy; 

of the 4 

of the 4 

of the 4 

describe 

knowledge: 


how to use 

listed in the 

listed in the 

listed in the 

any of the 4 

Questions 1, 


the 

column 

column 

column 

listed in the 

2, 3,4, 5, 6, 


strategy, 

High Level 

High Level 

High Level 

column 

7, and 8 


why the 
strategy 
was used in 
cognitive 
terms, and 
how they 
knew that 
was the 
strategy to 
use in 
cognitive 
terms. 

of Meta¬ 
cognitive 
Awareness. 

of Meta¬ 
cognitive 
Awareness. 

of Meta¬ 
cognitive 
Awareness. 

High Level 
of Meta¬ 
cognitive 
Awareness. 
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Planning 

Regulation 

The 

The 

The 

The 

The 

Questions 


participant 

participant 

participant 

participant 

participant 

1 and 2 


describes 

describes 2 

describes 2 

describes 1 

describes 1 



planning, 

of the items 

of the 3 

of the items 

of the 3 



goal 

in detail 

listed in the 

in detail 

listed in the 



setting, and 

and 1 item 

column 

and 2 items 

column 



allocation 

generally in 

High Level 

generally in 

High Level 



of 

the column 

of Meta- 

the column 

of Meta- 



resources 

High Level 

cognitive 

High Level 

cognitive 




of Meta- 

Awareness. 

of Meta- 

Awareness. 




cognitive 


cognitive 





Awareness. 


Awareness. 


Organization 

Regulation 

The 

The 

The 

The 

The 

Question 3 


participant 

participant 

participant 

participant 

participant 



describes 

describes 

describes 

describes 

does not 



the 

the 

the 

generally 

describe the 



implement¬ 

implement¬ 

implement¬ 

the 

implement¬ 



ation of 

ation of 

ation of 

implement¬ 

ation of 



techniques 

techniques 

techniques 

ation of 

techniques 



based on an 

based on an 

for the 

techniques 

for the 



under¬ 

under¬ 

purpose of 

for the 

purpose of 



standing of 

standing of 

organ¬ 

purpose of 

organ¬ 



cognition 

cognition 

ization. 

organ¬ 

ization. 



for the 

for the 


ization 




purpose of 

purpose of 






organ¬ 

organ¬ 






ization. 

ization. 




Information 

Regulation 

The 

The 

The 

The 

The 

management 


participant 

participant 

participant 

participant 

participant 

Question 4 


describes 

describes 3 

describes 2 

describes 1 

does not 



their 

of the 4 

of the 4 

of the 4 

describe 



cognitive 

items listed 

listed in the 

listed in the 

any of the 4 



organ¬ 

in the 

column 

column 

listed in the 



ization, 

column 

High Level 

High Level 

column 



elaboration, 

High Level 

of Meta- 

of Meta- 

High Level 



summar¬ 

of Meta- 

cognitive 

cognitive 

of Meta- 



ization, and 

cognitive 

Awareness. 

Awareness. 

cognitive 



selective 
focus on 
important 
information 

Awareness. 



Awareness. 

Monitoring 

Regulation 

The 

The 

The 

The 

The 

Questions 5 


participant 

participant 

participant 

participant 

participant 

and 6 


describes 

describes 2 

describes 2 

describes 1 

describes 1 



cognitive 

of the items 

of the 3 

of the items 

of the 3 



assessment 

in detail 

listed in the 

in detail 

listed in the 



of 

and 1 item 

column 

and 2 items 

column 



themselves, 

generally in 

High Level 

generally in 

High Level 



someone 

the column 

of Meta- 

the column 

of Meta- 



else, and 

High Level 

cognitive 

High Level 

cognitive 




of Meta- 

Awareness. 

of Meta- 

Awareness. 
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their 

cognitive 


cognitive 




strategy 

Awareness 


Awareness. 


Debugging 

Regulation 

The 

The 

The 

The 

The 

Question 7 


participant 

participant 

participant 

participant 

participant 



describes a 

describes 1 

describes a 

generally 

does not 



strategy 

of the items 

strategy 

describes 1 

describe a 



used to 

in detail 

used to 

of the 2 

strategy 



correct per¬ 

and 1 item 

correct per¬ 

aspects in 

used to 



formance 

generally in 

formance 

the column 

correct per¬ 



errors and 

the column 

errors or 

High Level 

formance 



assumption 

High Level 

assumpt¬ 

of Meta¬ 

errors or 



s they made 

of Meta¬ 

ions they 

cognitive 

assumpt¬ 



about a task 

cognitive 

made about 

Awareness. 

ions they 



or strategy 

Awareness. 

a task or 


made about 



used. 


strategy 


a task or 





used. 


strategy 

used. 

Evaluating 

Regulation 

The 

The 

The 

The 

The 

Question 8.2 


participant 

participant 

participant 

participant 

participant 



describes 

describes 1 

describes 

generally 

does not 



their own 

of the items 

their own 

describes 1 

describe 



post hoc 

in detail 

post hoc 

of the 2 

their own 



analysis of 

and 1 item 

analysis of 

aspects in 

post hoc 



their per¬ 

generally in 

either their 

the column 

analysis of 



formance 

the column 

per¬ 

High Level 

either their 



and 

High Level 

formance 

of Meta¬ 

per¬ 



strategy 

of Meta¬ 

or strategy 

cognitive 

formance 



effective¬ 

cognitive 

effect¬ 

Awareness. 

or strategy 



ness. 

Awareness 

iveness. 


effective¬ 







ness. 


The coding took at least two hours for each interview. After reading each 
section of the transcript, the coder assigned a level from zero (0) to five (5), 
based on the rubric, to the participant’s response. To help prevent miscoding a 
participant’s response, the coders wrote a description based on the coding rubric 
of areas that the participant did and did not address in their response to each 
question. The description was used to help identify if a participant’s response 
was correctly assigned a level based on the rubric. During coding, blue 
highlighters were used to code the regulation of cognition components, and 
yellow highlighters were used to code the knowledge of cognition components 
in each transcript. The coders also took notes about each participants’ 
metacognitive awareness level. These notes were used to help each coder write a 
synopsis about the participant’s overall level of metacognitive awareness. The 
synopses helped to form an apparent view of each participant’s metacognitive 
awareness. Coders also noted participant’s responses that were assigned higher 
levels based on the coding rubric or that responded uniquely. 

Using the assigned level from each coder on each interview question, 
interrater reliability was determined using Cohen’s kappa. Cohen’s kappa 
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compares the first and second coders’ assigned levels for all 18 participants and 
10 scored response sections in each interview. The 10 responses sections 
included participants’ responses to Questions 1-7 and Question 8.2, including 
the three parts of Question 4. Because Question 8.1 is a polar question requiring 
a yes or no answer, it was not scored by the coders. The interviewer attempted to 
keep the interviews to 1 hour each, and some participants were unable to answer 
all of the questions in the allotted time. Thus, the total number of compared 
items was 173 not 180. The interrater reliability analysis started by checking for 
miscodes on the transcripts. A miscode was when the coder’s assigned level and 
rationale for that level did not match. The miscodes required the coder to verify 
and correct their assigned level and rationale. After the miscodes were 
addressed, the data was entered into SPSS. The Cohen’s kappa statistic was then 
used to analyze the data for interrater reliability. Cohen’s kappa indicated that 
coders had a beyond-chance moderate agreement of 73%, a standard error of 
.03, and ap-value less than .001 in their assigned levels for the participants’ 
responses. 

The researcher compiled the coders’ codes, notes, and synopses for each 
participant into their respective group. Each group’s synopses consisted of 
participant’s assigned level of awareness on each of the 10 scored response 
sections from the interview, their responses that characterized the assigned level 
of awareness, and the coders’ notes and synopses. The combination of these 
items into a group summary helped characterize each group’s metacognitive 
awareness. After each group’s summary was complete, their unique 
metacognitive awareness themes became evident. 

Results 

Each of the three groups was found to have its own unique metacognitive 
awareness phenomenon while performing common teacher practices. The 
metacognitive phenomena of each group related to the many facets of the group 
members’ respective teacher practices. Regarding the knowledge of cognition 
component for all questions, only a few participants belonging to Groups 1 and 
3 exhibited medium to high levels of metacognitive awareness. Other than these 
participants, based on the coded transcripts, each group had a slightly different 
but overall low to medium level of metacognitive awareness in the knowledge of 
cognition component. The participants in each group frequently discussed their 
strategies (declarative knowledge) and infrequently discussed how to use the 
strategies for planning, organizing, monitoring, debugging, and evaluating 
(procedural knowledge). Occasionally, a participant from Groups 1 and 3 would 
also discuss why the strategy was used in terms of cognition. Listed below are 
representative examples of responses to interview Question 3 that were coded 
for the knowledge of cognition component. 
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Right now, the way 1 do things primarily is I will introduce the topic, and I 
can either be just showing slides, or a presentation, or 1 like to if I can show 
an audio or a video. (Group 3) 

Once they know the information then we will kind of go back and revisit all 
of that, but in the applicable sense where they’re actually going to be 
performing the task or the skill. (Group 3) 

I sit down with the standard course of study, create a pacing guide, roughly 
when I’m going to teach what, how much time it’s going to take, and then 
create lessons for each day. (Group 2) 

Typically, what I’ll do is 1 have a PowerPoint slide that I’ll throw up. It’ll 
give the day’s agenda for the three different sections I’ve been teaching for 
that period, and it’ll give specific warm ups. (Group 1) 

The participants from Groups 1 and 2 focused on a strategy and generally how 
to use the strategy for planning and organization. The participants’ responses 
from Group 3 included strategies, generally how to use the strategies, and a 
conditional reason why the strategy was used in terms of cognition. However, 
when comparing each group’s knowledge of cognition, the groups were similar. 
The groups’ transcripts had specific declarative knowledge, usually general 
procedural knowledge, and often lacked conditional knowledge. 

The groups’ answers to interview questions based on the regulation of 
cognition component were more distinguishably different. Group 3 participants 
were assigned more high and high-to-medium levels, Group 2 participants’ 
levels ranging from medium to low, and Group 1 participants were assigned 
more high-to-medium and medium levels. In two cases, a participant from 
Groups 1 and 2 was assigned a level of 0. The 0 level was not on the coding 
rubric, but both coders in both cases recorded a 0 for those participants’ 
responses. For Questions 1 and 2, coders were looking for the participant to 
describe planning, goal setting, and allocation of resources. Listed below are 
representative examples of the regulation of cognition coded responses from 
interview Question 1 dealing with the planning of instruction. 

With this unit, I used what the state had provided, but 1 also sought 
additional resources by using an opportunity to participate in a grant 
program. This program provided additional resources for the students to 
use. (Group 3) 

One of the methods 1 use for planning instruction is called the 5 E’s, and its 
engagement, evaluation, and it’s a couple more. (Group 2) 
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I draw on my past experience quite a bit. 1 take a look at what has worked in 
the past and maybe what hasn’t worked so well and then I formulate my 
lecture, my demonstrations based on that information. (Group 1) 

The passage from the Group 3 participant exemplifies how Group 3 members, 
those who had National Board Certification, often discussed goals or allocation 
of resources in addition to planning. Group 3 participants would often focus on 
two items from the rubric and only provide some detail on the third. The Group 
1 passage is primarily focused on planning. Group 1 often focused on planning 
with brief descriptions of either goal setting or allocation of resources. The 
passage from Group 2 presents a typical answer from participants in Group 2. 
The combination of incomplete thoughts and little detail on one or two of the 
rubric items resulted in low assigned levels of metacognitive awareness in the 
regulation of cognition component for interview Questions 1 and 2. 

Questions 3 and 4 were focused on organization. Question 4 was specific to 
information management, an aspect of organization. In Question 3, coders were 
looking for the participant to describe implementation of techniques based on an 
understanding of cognition for organization. Listed below are representative 
examples of the responses to interview Question 3 that were coded regulation of 
cognition. 

I would begin with some type of bell ringer. Something to get the students 
interested in what the topic is for that particular day. (Group 3) 

The overall objective for the lesson [was] to draw the kids interest, to help 
them make connections to the real world. (Group 3) 

I try to make them aware of where we are today, where we’ve come from, 
and hopefully where we’re going to go in the future. (Group 2) 

I usually start the class out with some sort of bell ringer, to get them 
thinking about what it is that we’re going to do that day. (Group 1) 

By taking it further and possibly doing a hand-on or application project with 
it, they’re involved. (Group 1) 

Group 3 participants, the National Board Certified Teachers, were focused on 
organization, attending to the cognitive needs of the students to keep them 
interested. Group 2 participants responded to Question 3 in general terms. The 
coders had difficulty highlighting any significant responses from Group 2 
participants. Due to Group 2’s generalized responses to Question 3, their focus 
seemed to be at the macro level of organization. Group 1 participants’ answers 
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to Question 3 were similar to those of Group 3. Occasionally, participants in 
Group 1 lacked the level of detail shown in Group 3’s responses. 

Question 4 was split into three scenarios dealing with information 
management. The first scenario was a general question about interpreting 
information. The second scenario was specific to information management of 
nonverbal feedback from students. The third scenario related to information 
management of new teaching techniques learned from another teacher or from a 
PD program. In all three scenarios, coders were looking for the participant to 
describe their cognitive organization, elaboration, summarization, and selective 
focus on important information. Listed below are representative examples of the 
responses to interview Question 4 that were coded as regulation of cognition. 

When I receive information, 1 try to make it relevant to what the situation is 
as far as my perspective, how I’m going to view it. (Group 3) 

When I receive information, I try to internalize it and fully understand the 
whole concept without just judgment about the information. (Group 3) 

I try to sometimes put myself in the position of being someone else. (Group 

3) 

I get information and sometimes 1 try to apply it to a project, maybe, the 
kids are working on. (Group 3) 

Based on your experience or based on your colleagues that you’re working 
with or talking to about it, you can evaluate some of the things as yeah, this 
is really important piece of information that they need to know, or maybe 
this particular piece of information is not as critical. (Group 2) 

I try to put it into some type of situation that I’m maybe familiar with. What 
am I going to do with this information, how am I going to apply it to 
something I already know or something that I need to know. (Group 1) 

Group 3 participants discussed how they organized information, focused on the 
important information, and either summarized or elaborated on the information 
that they received. The Group 2 participants mostly talked about one item 
specifically, and talked about the other aspects of information management more 
generally or not at all. In the Group 2 passage above, the participant’s entire 
response was specific to focusing on important information, but the participant 
neglected any real detail about other aspects of information organization. The 
Group 1 participant’s answers were more associated with the elaboration and 
summarization of information as well as the general management of 
information. 
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In Questions 5 and 6, the coders were looking for the monitoring 
subcomponent of cognitive regulation. Based on the rubric, for a participant’s 
response to be coded as a high level, they needed to describe cognitive 
assessment of each item in detail. All the participants at least described the 
monitoring of their students. However, Group 2 often did not describe the 
cognitive assessment portion. Group 2 discussed general monitoring of their 
students. Only a few participants described the cognitive assessment of 
themselves or their methods. Only three participants, one from Group 1 and two 
from Group 3, described in detail the cognitive assessment of themselves, 
someone else, and their strategy. Listed below are representative examples of 
the responses coded regulation of cognition from interview question five, which 
dealt with the monitoring of cognition during instruction. 

One team was asking the same question and another team was asking the 
same question, even though they’re two different ability levels then that 
means 1 missed it somewhere and I need to go back and cover that. (Group 

3) 

I will need to change my lesson plan just for that one level and 1 can 
remediate with them later or right at that moment. (Group 3) 

I walk my classroom and make sure that the students are doing individual 
work, or if they are working in learning teams then 1 know what they are 
doing. (Group 2) 

I kind of monitor myself because if what I’m saying doesn’t make sense to 
me . . . but I’ve already said something that after I say that, “wait a minute, 
what did 1 just say?” (Group 1) 

Question 7 focused on the debugging or adjustment of the teacher’s strategy 
during instruction. The coders were looking for the participant to describe a 
strategy used to correct performance errors and false assumptions that they had 
made about the task or strategy being used. None of the participants described 
the correction of both performance errors and false assumptions with enough 
detail for both coders to assign a high level of cognitive regulation in the 
debugging subcomponent. Group 1 provided the most detailed descriptions of 
their adjustments. Group 1 focused on either errors or assumptions with brief 
reference to the other aspect of the component. Group 2 received the lowest 
levels on this component. Both coders wrote that the responses of Group 2 
participants were general and sometimes did not seem to provide a direct answer 
to Question 7. Group 3 answered similarly to Group 1 but lacked the same level 
of detail. The coders considered Group 3 participants to be more at the medium 
level of metacognitive awareness for the debugging subcomponent based on the 
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coding rubric. Listed below are representative examples of the regulation of 
cognition coded responses from interview Question 7. 

I adjust my teaching during the lesson based on how 1 think the lesson is 
going . . . whether it’s contextual feedback or if it’s a spoken-type of 
feedback or body language. I adjust my lesson just based on what I see if 
it’s working or not. (Group 3) 

I’m trying to make sure that I’m not losing some of them. If I do I try to 
back up and show a different way to do a particular thing. (Group 2) 

I might backtrack and re-explain something. 1 might try a different way of 
explaining something . . . switch people to different groups ... I might try 
to find a totally different way to explain something. Some groups, 1 might 
do hands-on activities. Other groups, 1 might show a video or see a 
PowerPoint, just depending on the dynamics. (Group 1) 

Part two of Question 8 was about the participants’ cognitive self-evaluation 
and reflection. The coders were looking for the participants to describe their 
own post hoc analyses of their performance and strategy effectiveness. Based on 
the coding of their responses, the groups all had similar metacognitive 
awareness on question eight; they were all basically assigned at a medium level 
of metacognitive awareness. Some of the participants provided somewhat more 
detail than others, resulting in slightly higher levels of awareness being 
assigned. All participants focused on reflecting either about their performance or 
their strategy effectiveness. The participants described how, when, and why they 
tended to reflect. Some of the participants even described what they did based 
on decisions made during their self-evaluation. Listed below are representative 
examples of the regulation of cognition coded responses from interview 
Question 8.2. 

I go back and look at what didn’t work, what I need to change. (Group 3) 

The next time 1 teach this lesson. I’m going to do this. 1 might leave this 
part off. That’s basically how I plan. (Group 3) 

You have to look back on it, and say, “well, that really didn’t go well that 
way, next time I can try it this way.” (Group 2) 

A lot of times driving home I’ll think about what 1 did that day. Did it 
work? How can I do something a little different? Make it a little better, 
make it a little more interactive. (Group 2) 
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1 try to make little notes to myself about what went right, what went wrong, 
come up with ideas on how I could change things and do thing differently. 
(Group 1) 


Implications 

The first finding from this study indicated that all three groups had similar 
levels of metacognitive awareness in the knowledge of cognition component. 
Coders indicated based on the coding rubric that participants typically ranged 
from medium to low levels of cognitive knowledge. The literature, especially in 
technology and engineering related PD, has discussed the importance of content 
and pedagogical knowledge in PD (Bybee & Loucks-Horsley, 2000; Daugherty 
& Custer, 2012; Mundry, 2007; WestEd, 2000). Content and pedagogical 
knowledge are often considered foundational characteristics of effective PD 
(Ball, Thames, & Phelps, 2008; Bybee & Loucks-Horsley, 2000; Mundry, 

2007). Underlying both content and pedagogical knowledge is knowledge of 
cognition; the knowledge of strategies, skills, and abilities that impacts a 
person’s learning ability as well as how and why to use strategies and techniques 
for increasing performance and accomplishing cognitive tasks. Moreover, research 
indicates that teachers’ knowledge of cognition is linked to their learning ability, 
pedagogical effectiveness, ability to transfer learning from one context to 
another, and ability to adapt in a complex educational environment (Bransford et 
al., 2000; Ertmer & Newby, 1996; Georghiades, 2004; Gourgey, 1998; Hartman, 
2001; Kramarski & Michalsky, 2009; Lin et al., 2005; Pucheu, 2008; Wilson & 
Bai, 2010). This would suggest that teacher PD should focus on metacognitive 
awareness, including knowledge of cognition, because it will impact many 
common teacher practices. 

The second finding from this study indicated that each group had uniquely 
different metacognitive awareness in the regulation of cognition component. 
Overall, Groups 1 and 3 had higher levels in regulation of cognition component, 
and these two groups successfully completed their PD experience. Self¬ 
regulation is considered an aspect within the regulation of cognition component 
of metacognition (Ertmer & Newby, 1996; Pintrich, Wolters, & Baxter, 2000). 
Self-regulation has been identified as a factor in a person actively pursuing the 
learning process (Ertmer & Newby, 1996; Pintrich et al., 2000). Regulation of 
cognition is a decisive factor in teachers’ adaptation ability, which helps in 
solving problems involving information management and reasoning (Hartman, 
2001; Kramarski & Michalsky, 2009; Lin et al., 2005). The complexity that 
teachers experience each day necessitates their ability to regulate their cognition 
(Hartman, 2001; Kramarski & Michalsky, 2009; Lin et al., 2005). Regulation of 
cognition’s role in teachers’ learning, ability to adapt, and development 
throughout their career suggests its importance as a focus in PD programs. 
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Conclusions 

The intent of this research was to understand technology and engineering 
teachers’ level of metacognitive awareness in comparison to their participation 
and completion of either the T2I2 or the NBPTS program. The findings suggest 
a connection between teachers’ level of metacognitive awareness, successful 
completion of PD, and ability to manage educational complexity. The results of 
this study are applicable to future work in improving teacher content and 
pedagogical knowledge, helping teachers manage educational complexity, 
helping teachers take an active self-regulated role in PD, and PD effectiveness. 
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